Method of controlling skeleton model, and recording medium therewith

ABSTRACT

A method of controlling a model by a computer, includes accepting an instruction of changing a pose or a movement of a standard model for which control data for joint angles is attached; determining a pose or a movement of the standard model based on the instruction and the control data for joint angles; and determining a pose or a movement of a target model to be controlled where parts of the standard model and parts of the target model being in correspondence with each other, such that the pose or the movement of the target model follows the pose or the movement of the determined standard model based on the correspondence of the parts of the standard model and the parts of the target model.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates a method of controlling a model.

2. Description of the Related Art

When designing a pose or animation by moving a CG character or the like, a skeleton model in which plural bones are connected by joints is used.

FIG. 1A is a view showing an example of such a skeleton model 100. The skeleton model 100 includes a first bone 110 (a parent bone or a parent node), a second bone 112 (a child bone or a child node) connected to the first bone 110 via a joint 114, for example.

The skeleton model 100 can be expressed by nodes connected in a parent-child relationship as shown in FIG. 1A. The bone is a line segment connecting the origins of the nodes in the parent-child relationship, or the node itself.

A status of rotation of the joint 114 (bending, torsion) can be expressed by a direction of the second bone 112 (child node) with respect to the first bone 110 (parent node). In order to express the direction of the node, a coordinate system is defined for each of the nodes.

Such a skeleton model may be used not only in a virtual space but also in a real space for controlling a target such as a robot or the like. For such a skeleton model which is used when controlling a posture (pose, movement or the like) of a human body model, an animal model, a fabric or the like on a three dimension (3D) virtual space or a real space, it is necessary to control a position, an angle, rotation (bending, torsion) or the like of each of the bones and joints to not be unnatural. As shown in FIG. 1A, in the skeleton model 100, the bones are connected by the joints so that the movement of each of the bones influences the movement of another bone.

For example, the skeleton model 100 shown in FIG. 1A is a skeleton model of a human body. Thus, there should be a limitation in the movement of an arm with respect to an elbow. If the skeleton model 100 takes a pose in which the arm of the skeleton model 100 is bent with an angle beyond this limitation, the skeleton model 100 becomes unnatural far from an actual human body. Thus, in order to form a skeleton model which takes a natural pose like a human body, it is necessary to set a certain limitation to the movement of a bone corresponding to an arm with respect to a joint corresponding to an elbow.

Patent Document 1 discloses a skeleton model in which a direction of a child bone with respect to a parent bone is expressed by rotation angles around three axes (x, y, z), respectively, in a coordinate system of the parent bone.

In Patent Document 2, a technique is disclosed in which a predetermined point on a spherical surface having a joint as a center is set as a focal point, a surface which is perpendicular to an axis connecting the center and the focal point is set as a projecting surface, and the movable range of the joint is set on the projecting surface.

In Patent Document 3 and Patent Document 4, a technique is disclosed in which a direction of a child bone with respect to a parent bone is corrected, when the direction of the child bone is not within a predetermined movable range, in order to satisfy the condition corresponding to the movable range.

In Patent Document 5, a technique is disclosed in which a calculated amount is reduced by using a monopolar coordinate system when moving a revolute skeleton model at real-time.

In Patent Document 6, a technique is disclosed in which hardness in bending joints of a revolute model is controlled.

However, according to the techniques disclosed in Patent Document 1 to Patent Document 6, when a new skeleton model is generated, it is necessary to set an angle limitation or hardness in bending for each of the joints. This setting is troublesome for users. Further, in order to set the angle limitation or hardness in bending to have the new skeleton model to take a real moving pose, knowledge and experience are required.

Patent Document

-   [Patent Document 1] Japanese Laid-open Patent Publication No.     2000-306116 -   [Patent Document 2] Japanese Laid-open Patent Publication No.     2009-070340 -   [Patent Document 3] Japanese Laid-open Patent Publication No.     2010-170279 -   [Patent Document 4] US published patent application No. 2010-182329 -   [Patent Document 5] Japanese Patent Application No. 2011-024449 -   [Patent Document 6] Japanese Patent Application No. 2011-025361

SUMMARY OF THE INVENTION

The present invention is made in light of the above problems, and provides a method of controlling a posture of a skeleton model capable of reducing a troublesome method of setting angle limitations of a joint, hardness in bending joints or the like.

According to an embodiment, there is provided method of controlling a model by a computer, including: accepting an instruction of changing a pose or a movement of a standard model for which control data for joint angles is attached; determining a pose or a movement of the standard model based on the instruction and the control data for joint angles; and determining a pose or a movement of a target model to be controlled where parts of the standard model and parts of the target model being in correspondence with each other, such that the pose or the movement of the target model follows the pose or the movement of the determined standard model based on the correspondence of the parts of the standard model and the parts of the target model.

According to another embodiment, there is provided a method of determining a pose or a movement of a target model to be controlled including plural groups of bones each group including one or more bones, using a standard model including plurality of groups of bones each including one or more bones, the groups of bones of the standard model and the groups of bones of the target model being in correspondence with each other, including specifying a group of bones of the standard model and a corresponding group of bones of the target model based on the correspondence of the groups of bones of the standard model and the groups of bones of the target model, determining a direction of the specified group of bones of the target model based on the specified group of bones of the standard model; and determining a direction of each of the bones included in the specified group of bones of the target model based on the corresponding bones included in the specified group of bones of the standard model.

Note that also arbitrary combinations of the above-described constituents, and any exchanges of expressions in the present invention, made among methods, devices, systems, recording media, computer programs and so forth, are valid as embodiments of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features and advantages of the present invention will become more apparent from the following detailed description when read in conjunction with the accompanying drawings.

FIG. 1A is a view showing an example of a skeleton model;

FIG. 1B is a view showing an example of a part of the skeleton model;

FIG. 2A is a view showing an example of a template;

FIG. 2B is a view showing an example of a character;

FIG. 3 a view showing an example of coordination of bone groups according to an embodiment;

FIG. 4 is a flowchart showing an example of an operation of determining correspondence between bone groups of template and character;

FIG. 5 is a view for explaining a rotation of a bone;

FIG. 6A to FIG. 6C are views for explaining a method of calculating a coordinate system adjusting matrix;

FIG. 7 is a flowchart showing an operation of controlling a model of an embodiment;

FIG. 8 is a block diagram showing an example of a hardware structure of a model control system;

FIG. 9 is a view showing two corresponding bone groups of the character and the template;

FIG. 10 is a view for explaining a method of obtaining a rotation matrix for obtaining an overlapped state;

FIG. 11 is a view showing two coordinate systems of the character and the template;

FIG. 12 is a view showing states of bones in bone groups;

FIG. 13 is a view showing an example of current template bones in a bone group coordinate system;

FIG. 14 is a view for explaining an operation of determining a pose of a character;

FIG. 15 is a view showing an example of a template skeleton model and a character skeleton model at an initial state;

FIG. 16 is a view showing a simple rotation for coinciding axes;

FIG. 17 is a view showing a method of interpolating;

FIG. 18 is a view showing adjusting a bone axial vector;

FIG. 19 is a view showing a simple rotation;

FIG. 20A to FIG. 20C are views showing a difference between skeleton models of a first example and a second example;

FIG. 21 is a view showing a difference between skeleton models of a first example and a second example; and

FIG. 22 is a block diagram showing an example of a functional structure of a model control system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will be described herein with reference to illustrative embodiments. Those skilled in the art will recognize that many alternative embodiments can be accomplished using the teachings of the present invention and that the invention is not limited to the embodiment illustrated for explanatory purposes.

It is to be noted that, in the explanation of the drawings, the same components are given the same reference numerals, and explanations are not repeated.

Words used in this embodiment are defined as follows. These words are further explained in detail later.

-   (1) A “skeleton” means a bone or the like. -   (2) A “node” means each portion of a model. -   (3) A “bone” means a line segment connecting origins of two nodes     which are in a parent-child relationship, or a node itself. -   (4) A “bone group” means an assembly of linearly connected one or     more bones. -   (5) An “object” means a target skeleton model for which a pose or a     movement is to be determined. -   (6) An “object coordinate system” means a coordinate system of a     highest parent node (root node). -   (7) A “local coordinate system” is a coordinate system of a parent     bone or a parent bone group. -   (8) A “template” means a standard skeleton model with control data     for joint angles. -   (9) A “character” means a skeleton model to be controlled without     control data for joint angles.

Other words are explained later.

FIG. 2A is a view showing an example of a template 120. FIG. 2B is a view showing an example of a character 130.

FIG. 3 is a view showing an example of a standard skeleton model 122 (standard model) and a target skeleton model 132 (target model) to be controlled. The template 120 shown in FIG. 2A includes the standard skeleton model 122, and the character 130 shown in FIG. 2B includes the target skeleton model 132.

In this embodiment, control data for joint angles are not set for the target skeleton model 132 of the character 130.

However, control data for joint angles are previously set for the standard skeleton model 122 of the template 120. The control data for joint angles includes a setting of an angle limitation or hardness in bending for each of the joints.

The standard skeleton model 122 of the template 120 is a previously prepared skeleton model having a function to apply a limitation to a pose or a movement of the character 130 shown in FIG. 2B to restrict an unnatural pose or movement of the character 130.

In this embodiment, in order to change a pose or a movement of the character 130 (FIG. 2B), a pose or a movement of the template 120 (FIG. 2A) is changed first. As the template 120 is provided with the control data for joint angles, the template 120 is controlled not to take an unnatural pose or movement.

Actually, an operator inputs an instruction causing a pose or a movement of the character 130 via a user interface. The instruction is applied to the template 120 shown in FIG. 2A first.

As will be explained later, parts of the template 120 and parts of the character 130 are in correspondence with each other.

Thus, specifically, the instruction of changing the pose or the movement of the character 130 is converted to the instruction of changing the pose or the movement of the target 120 based on the correspondence of the parts of the template 120 and the parts of the character 130. Then, the pose or the movement of the target 120 is determined.

Then, the pose or the movement of the template 120 is copied for the character 130 so that a new pose or movement for the character 130 is displayed.

In the above series of operations, it is not necessary to display the template 120 on a screen. This means that the operator may input the instruction of the pose or the movement just seeing an image of the character 130. With this user interface, the operator can instruct providing a natural pose or movement of the character 130 without recognizing existence of the template 120.

However, the embodiment is not limited so, and the template 120 may also be displayed. Further, a user interface in which the operator can operate the template 120 may be provided.

When the template 120 is displayed, the operator may input an instruction of changing the pose or the movement of the template 120 in order to change the pose or the movement of the character 130 at last.

The standard skeleton model 122 and the target skeleton model 132 include plural bone groups, respectively as shown in FIG. 3. The standard skeleton model 122 includes a bone group A′ and a bone group B′. The target skeleton model 132 includes a bone group A and a bone group B. In this embodiment, the bone groups of the target skeleton model 132 are in correspondence with the bone groups of the standard skeleton model 122. Specifically, the bone group A′ and bone group B′ of the standard skeleton model 122 are in correspondence with the bone group A and bone group B of the target skeleton model 132, respectively.

Even when the numbers of the bones included in the skeleton models are different, the bone groups can be in correspondence with each other.

The bone groups of the standard skeleton model 122 and the target skeleton model 132 may be determined to correspond with each other using a known technique of pattern recognition. Alternatively, the operator may previously designate the correspondence of the bone groups. Further alternatively, the operator may correct the correspondence of the bone groups obtained by the technique of pattern recognition.

Plural of the templates 120 may be prepared. Plural kinds of templates of a human body model, an animal model, a fabric or the like may be prepared. In such a case, the operator may select an appropriate template among the plural templates. Alternatively, an appropriate template may be selected by the known technique of pattern recognition based on the character 130.

As described above, the selection of the template 120 is performed and the correspondence of the bone groups is determined.

Then, the operator inputs an instruction to change the pose or the movement of the character 130 via an appropriate user interface. Upon accepting the instruction, a system, which will be explained later with reference to FIG. 8 and FIG. 22, changes the pose or the movement of the template 120, not the character 130. As described above, the template 120 is provided with the control data for joint angles so that the template 120 does not take the unnatural pose or movement. Then, the system copies the pose or the movement of the template 120 into the character 130. Then, the system displays the pose or the movement of the character 130 with appropriate time intervals.

When the target skeleton model 132 of the character 130 includes a bone group including plural bones, it can be controlled such that the plural bones included in the bone group smoothly move based on the status of the nodes upstream and downstream of the bone group.

FIG. 8 is a block diagram showing an example of a hardware structure of a model control system 800.

The model control system 800 includes a CPU 801, a memory 802, an input-output device 803, a display device 804, and a printer 805. The input-output device 803 may be a mouse, a tablet, a touch screen, a keyboard, a scanner or the like.

FIG. 22 is a block diagram showing an example of a functional structure of the model control system 800. The memory 802 includes a static memory and a dynamic memory. The model control system 800 may be a computer.

The model control system 800 includes an input unit 850, a correspondence determining unit 852, a template pose determining unit 854, a character pose determining unit 856, a template storing unit 860, and a character storing unit 862.

The template storing unit 860 stores the template 120 (the standard skeleton model 122) with the control data for joint angles. The template storing unit 860 may store plural kinds of templates. The character storing unit 862 stores the character 130 (the target skeleton model 132).

The input unit 850 accepts an instruction from an operator via the input-output device 803. The correspondence determining unit 852 determines the correspondence between the bone groups of the template 120 and the character 130. The determined correspondence may be stored in the character storing unit 862 with the character 130.

The template pose determining unit 854 determines the pose of the template 120 based on the instruction from the operator and the control data for joint angles. When the pose selected to be used by the operator is a natural pose based on the control data for joint angles, the template pose determining unit 854 determines to change the pose of the template 120 as instructed. However, when the pose selected by the operator is an unnatural pose or the like, the template pose determining unit 854 determines not to change the pose of the template 120 and may display an error notice or the like for the operator via the display device 804. In this embodiment, the pose selected by the operator is modified based on the control data for joint angles for the pose of the template 120.

The character pose determining unit 856 determines the pose of the character 130 by copying the pose of the character 120 determined by the template pose determining unit 854. Then, the character 130 with the determined pose is displayed on the display device 804.

(Correspondence of Bone)

First, an operator generates a character and selects a template corresponding to the generated character. It is assumed that the character 130 as shown in FIG. 2B is generated and the template 120 as shown in FIG. 2A is selected.

Bone groups are generated for the template 120 and the character 130. Then, whether the bone groups of the template 120 and the character 130 are in correspondence with each other is determined.

A pair of bone groups of the template 120 and the character 130 is searched as follows, for example.

FIG. 4 is a flowchart showing an example of an operation of determining correspondence between bone groups of the template 120 and the character 130. This operation is performed by the correspondence determining unit 852 (see FIG. 22).

In step S402, bones of the template 120 and bones of the character 130 are determined to be in correspondence with each other, respectively, by a known technique of pattern recognition or the like. Then, bone groups are generated from the bones for the template 120 and the character 130, respectively, and the generated bone groups are in correspondence with each other.

In step S404, whether all of the bones of the template 120 and the character 130 belong to the bone groups is determined. When it is determined that all of the bones belong to the bone groups (YES in step S404), the process ends.

When it is determined that not all of the bones belong to the bone groups (NO in step S404), the process proceeds to step S406.

In step S406, one of the bones which does not belong to any of the bone groups is selected.

In step S408, whether the bone groups exist at both sides of the selected bone is determined. When there is a bone group at one side, the selected bone is added to the bone group. When there are bone groups at both sides, the selected bone is added to one of the bone groups. Which of the bone groups is selected to include the bone may be determined by a known technique of pattern recognition. When there are not bone groups at both sides, the selected bone is released. Then, the process moves back to step S404.

By performing the above operation, the bone groups are generated for the template 120 and the character 130 and are in correspondence with each other. The operator may correct the correspondence of the bone groups. Alternatively, the operator may designate the correspondence of the bone groups. The correspondence of the bone groups is not limited to the above embodiment.

(Calculation of Coordinate System by Matrix)

FIG. 5 is a view for explaining a rotation of a bone. In a three dimensional coordinate system xyz, when rotating an end “A” of a bone “PA” while having an origin “P” as a center and keeping the length of the bone “PA” constant to a position “A′”, coordinates of the position “A′” (x₀′, y₀′, z₀′) can be expressed as follows using a matrix M. Here, coordinates of the position “A” are (x₀, y₀, z₀).

$\begin{matrix} {\mspace{79mu}{{M = \begin{pmatrix} m_{00} & m_{01} & m_{02} \\ m_{10} & m_{11} & m_{12} \\ m_{20} & m_{21} & m_{22} \end{pmatrix}}{\left( {x_{0}^{\prime},y_{0}^{\prime},z_{0}^{\prime}} \right) = {{\left( {x_{0},y_{0},z_{0}} \right)M} = {\left( {x_{0},y_{0},z_{0}} \right)\begin{pmatrix} m_{00} & m_{01} & m_{02} \\ m_{10} & m_{11} & m_{12} \\ m_{20} & m_{21} & m_{22} \end{pmatrix}}}}}} & \left( {{Equation}\mspace{14mu} 1} \right) \end{matrix}$

Although rotation of a bone is exemplified in the above example, a relative relationship between two coordinate systems can be expressed by the matrix M. Further, each of the bones may have a unique coordinate system. In this embodiment, it is assumed that each of the bones has an individual (unique) coordinate system (see FIG. 1B).

For example, by obtaining a matrix Ma which converts unit vectors in x-axis, y-axis, and z-axis directions of a coordinate system of a bone to unit vectors in x-axis, y-axis, and z-axis directions of a coordinate system of another bone, the matrix Ma can express a relative relationship between these coordinate systems. In this embodiment, the matrix Ma is referred to as a “coordinate conversion matrix”.

Further, when rotating the end “A” of the bone “PA” while having the origin “P” as a center and changing the length of the bone “PA” to be longer to a position “A″”, coordinates of the position “A″” (x₀″, y₀″, z₀″) can be expressed as follows using a matrix Mt.

$\begin{matrix} {\mspace{79mu}{{{Mt} = \begin{pmatrix} m_{00} & m_{01} & m_{02} & 0 \\ m_{10} & m_{11} & m_{12} & 0 \\ m_{20} & m_{21} & m_{22} & 0 \\ a & b & c & 1 \end{pmatrix}}{\left( {x_{0}^{''},y_{0}^{''},z_{0}^{''},1} \right) = {{\left( {x_{0},y_{0},z_{0},1} \right){Mt}} = {\left( {x_{0},y_{0},z_{0},1} \right)\begin{pmatrix} m_{00} & m_{01} & m_{02} & 0 \\ m_{10} & m_{11} & m_{12} & 0 \\ m_{20} & m_{21} & m_{22} & 0 \\ a & b & c & 0 \end{pmatrix}}}}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack \end{matrix}$

Here, a, b, and c in the matrix Mt are referred to as a “trans unit”. The trans unit means a movement amount of a bone. When the matrix is a local matrix, the transfer unit expresses a movement amount from a parent, in other words, a length of the bone. On the other hand, “m₀₀” to “m₂₂” only contribute to rotation of the bone and are referred to as a “rotation unit”.

[Copy of Pose]

Copying of the pose is performed on the corresponding bone groups. In this embodiment, two examples are explained about the copying of pose, however, the copying of the pose is not limited solely to these two examples.

(1) Calculation of Coordinate System Adjusting Matrix

Although similar to the coordinate systems of the bones, the coordinate systems of the bone groups for the template 120 and the character 130 are different. Thus, first, a “coordinate system adjusting matrix” that expresses a relationship between the coordinate systems of the bone groups for the template 120 and the character 130 is calculated based on information about the bone group of the character 130 (which will be referred to as a “character bone group”) and the bone group of the template 120 (which will be referred to as a “template bone group”) at an initial state. This operation is performed once at first.

The method of calculating the coordinate system adjusting matrix is explained with reference to FIG. 6A to FIG. 6C.

(a) Generating Overlapped State

A directed line segment from a starting point to an end point of a bone group is referred to as a “bone group axis”.

It is necessary to have directions of the bone group axes of the template 120 and the character 130 become the same in order to calculate the coordinate system adjusting matrix.

First, using the template 120 and the character 130 at an initial state, the bone group axis of the character 130 is adjusted to coincide the bone group axis of the template 120.

FIG. 6A shows a part of the standard skeleton model 122 of the template 120 and FIG. 6B shows a part of the target skeleton model 132 of the character 130 at the initial state, respectively. The bone group B′ shown in FIG. 6A and the bone group B shown in FIG. 6B are in correspondence with each other, in other words, these are a pair. The bone group B′ includes a bone group axis α and a coordinate system e. The bone group B includes a bone group axis β and a coordinate system e′.

In this case, first, a matrix M1 for coinciding the direction of the bone group axis β to the direction of the bone group axis α is calculated.

Then, the bone group B of the target skeleton model 132 is actually rotated by using the matrix M1 as shown in FIG. 6C. Here, the bone group B comes to include a bone group axis β′ and a coordinate system e″ where the direction of the bone group axis β′ coincides the direction of the bone group axis α of the standard skeleton model 122. The bone group B at this state is referred to as in an “overlapped state”.

(b) Calculating Coordinate System Adjusting Matrix

Although the directions of the bone group axes of the character 120 at the overlapped state and the template 130 at the initial state coincide with the above operation, the coordinate systems (the coordinate system e and the coordinate system e″) of them are different. Thus, a matrix M2, which is the coordinate system adjusting matrix, for adjusting the difference of these coordinate systems is obtained.

The method of obtaining the coordinate system adjusting matrix is explained later in detail.

(2) Designating and Copy Pose or Movement

First, the system 800 accepts a designation from the operator to change the pose or the movement of the character 130.

Then, the model control system 800 changes the pose or the movement of the template 120. This operation is performed by the template pose determining unit 854.

The model control system 800 copies the pose or the movement of the template 120 into that of the character 130. The pose of the character 130 is determined by applying the coordinate system adjusting matrix of the pose of the template 120. Further, a status in the bone group is adjusted in accordance with necessity. With the above operation, the pose or the movement of the template 120 is copied into the character 130. This operation is performed by the character pose determining unit 856.

FIG. 7 is a flowchart showing an operation of the embodiment.

In step S702, the system 800 searches for a correspondence between the bone groups of the template 120 and the bone groups of the character 130. Alternatively, the correspondence between the bone groups may be designated by the operator.

In step S704, instruction to change the pose or the movement of the template 120 is accepted.

In step S706, the pose or the movement of the template 120 is determined by applying the control data for joint angles based on the instruction. As described above, when the pose or the movement of the template 120 selected by the operator is unnatural based on the control data for joint angles, the pose or the movement of the template 120 is not changed and an error message or the like is displayed for the operator.

In this embodiment, the pose selected by the operator is modified based on the control data for joint angles for the pose of the template 120.

In step S708, the pose or the movement of the character 130 is determined from the determined pose or movement of the template 120 using the correspondence between the bone groups.

With the above operation, the character 130 having a predetermined pose or movement is finally displayed.

(First Example and Second Example)

The first example and the second example are explained.

(1) First Example

Information whether the template 120 and the character 130 are having the same pose is not provided. Thus, the pose of the character 130 is changed in accordance with the pose of the template 120 while assuming that the template 120 and the character 130 are having the same pose when the directions of the bone axis and the bone group axis of the character 130 coincide the directions of the bone axis and the bone group axis of the template 120.

(2) Second Example

The template 120 and the character 130 having the same pose are provided as an initial state. Thus, the pose of the character 130 is changed in accordance with the pose of the template 130 while maintaining the relative relationship between the directions of the bone axis and the bone group axis of the character 130 and the directions of the bone axis and the bone group axis of the template 120 at the initial state.

The first example and the second example are explained in detail.

Parts of each of the skeleton models are connected in a parent-child relationship. Each of the parts of the skeleton models is referred to as a “node”. A “bone” means a line segment connecting origins of two nodes which are in a parent-child relationship, or a node itself. The node which is at a parent side is referred to as a “starting point” and the node which is at a child side is referred to as an “end point”.

For example, it is assumed that a child of a node n is a node n+1, and a bone generated by connecting these nodes is a bone m. At this time, the starting point of the bone m is the node n, and the end point is the node n+1. A directed line segment from the starting point to the end point is referred to as a “bone axis”.

Similar to the parent-child relationship of the nodes, bones have a parent-child relationship. For the above example, the bone in which the node n is the end point becomes a parent of the bone m. The bone in which the node n+1 is the starting point becomes a child of the bone m.

In the first example, a coordinate system of the end point of the bone is assumed as a “coordinate system of the bone” where an associated object of the bone is expressed by a coordinate system of a parent bone of the bone.

On the other hand, in the second example, a coordinate system of the starting point of the bone is assumed as a “coordinate system of the bone” where an associated object of the bone is expressed by the coordinate system of the bone.

FIG. 20A is a view showing a part of a skeleton model including a standard node with a coordinate system O, a child node of the standard node with a coordinate system C1, and further a child node with a coordinate system C2.

For the first example, as shown in FIG. 20A, the skeleton model includes two bones 200 and 202 from the parent side. A coordinate system of the bone 200 is the coordinate system C1 which is at the end point of the bone 200. Similarly, a coordinate system of the bone 202 is the coordinate system C2 which is at the end point of the bone 202.

In FIG. 20B, actual nodes n1 and n2 (body portions of the skeleton model) are also overlapped on the skeleton model shown FIG. 20A. The coordinate system of the node n1 is the coordinate system C1 and the coordinate system of the node n2 is the coordinate system C2.

For the second example, as shown in FIG. 20C, the skeleton model includes two bones 204 and 206 from the parent side. A coordinate system of the bone 204 is the coordinate system C1 which is at the starting point of the bone 204.

These differences of the examples are explained with reference to FIG. 21, in which an arm of a skeleton model 210 is shown as an example.

The skeleton model 210 includes a shoulder “a”, a brachium “b”, an elbow “c”, a forearm “d”, a wrist “e” and a hand “f”. The brachium “b”, the forearm “d” and the hand “f” are nodes, respectively. The shoulder “a” is the origin of the node of the brachium “b”, the elbow “c” is the origin of the node of the forearm “d”, and the wrist “e” is the origin of the node of the hand “f”. As shown in FIG. 21, the shoulder “a” to the elbow “c” is referred to as a bone 1, and the elbow “c” to the wrist “e” is referred to as a bone 2.

For the first example, the coordinate system of the bone 1 is determined as the coordinate system of the forearm “d” whose origin is the elbow “c”, and the coordinate system of the bone 2 is determined as the coordinate system of the hand “f” whose origin is the wrist “e”.

On the other hand, for the second example, the coordinate system of the bone 1 is determined as the coordinate system of the brachium “b” whose origin is the shoulder “a”, and the coordinate system of the bone 2 is determined as the coordinate system of the forearm “d” whose origin is the elbow “c”.

1. First Example in Detail

1.1. Node, Bone and Bone Group

(1.1.1. Node and Bone)

As described above, the coordinate system of the end point of the bone is determined as the “coordinate system of the bone” where the associated object of the bone is expressed by a coordinate system of a parent bone of the bone.

(1.1.2. Bone Group)

The bone group means an assembly of linearly connected one or more bones. Here, a connecting portion of two bones needs to be a starting point of one of the bones and an end point of the other of the bones. One end of the bone group at a parent side is referred to as a “starting point of the bone group”, and another end of the bone group at a parent side is referred to as an “end point of the bone group”.

The bone groups have a parent-child relationship similar to that of the bones.

The coordinate system of an end bone is used as the coordinate system of the bone group.

The n-th bone group of the template 120 corresponds to the n-th bone group of the character 130. These corresponding bone groups of the template 120 and the character 130 are referred to as a “bone group pair”.

1.2. Definition of Symbols, Basic Equations or the Like

(1.2.1. Common Symbols)

In the following, “′” means it is related to the template 120, and without “′” means it is related to the character 130.

“^((ini))” means an initial state (initial pose) of a bone, and without “^((ini))” means a current state (current pose) of a bone. “_(G)” means a bone group.

(1.2.2. Bones of Character and Template)

It is assumed that an n-th bone of the character 130 is referred to as a character bone CB_(n). Further, a parent character bone of the character bone CB_(n) is referred to as CB_(n−1).

L_(n) means a local matrix of the character bone CB_(n) that expresses a position (trans) and rotation (rotate) of the character bone CB_(n) in the CB_(n−1) coordinate system.

M_(n) means an object matrix of the character bone CB_(n) that expresses a position (trans) and rotation (rotate) of the character bone CB_(n) in the object coordinate system (=node 0 origin). M_(n)=L_(n)L_(n−1) . . . L₁=L_(n)M_(n−1)

L_(n) ^((ini)) means the local matrix “L_(n)” of the character bone CB_(n) at the initial state of the character 130.

M_(n) ^((ini)) means the object matrix “M_(n)” of the character bone CB_(n) at the initial pose of the character 130. M_(n) ^((ini))=L_(n) ^((ini))L_(n−1) ^((ini)) . . . L₁ ^((ini))=L_(n) ^((ini))M_(n−1) ^((ini))

Similarly, it is assumed that an n-th bone of the template 120 is referred to as a template bone TB_(n). Further, a parent template bone of the template bone TB_(n) is referred to as TB_(n−1).

L′_(n) means a local matrix of the template bone TB_(n) that expresses a position (trans) and rotation (rotate) of the template bone TB_(n) in the TB_(n−1) coordinate system.

M′_(n) means an object matrix of the template bone TB_(n) that expresses a position (trans) and rotation (rotate) of the template bone TB_(n) in the object coordinate system (=node 0 origin). M′_(n)=L′_(n)L′_(n−1) . . . L′₁=L′_(n)M′_(n−1)

L′_(n) ^((ini)) means the local matrix “L′_(n)” of the template bone TB_(n) at the initial pose of the template 120.

“M′_(n) ^((ini))” means the object matrix “M′_(n)” of the template bone TB_(n) at the initial pose of the template 120.

$\begin{matrix} {M_{n}^{\prime{({ini})}} = {L_{n}^{\prime{({ini})}}L_{n - 1}^{\prime{({ini})}}\mspace{14mu}\ldots\mspace{14mu} L_{1}^{\prime{({ini})}}}} \\ {= {L_{n}^{\prime{({ini})}}M_{n - 1}^{\prime{({ini})}}}} \end{matrix}$ (1.2.3. Bone Groups of Character and Template)

It is assumed that an n-th bone group of the character 130 is referred to as a character bone group CBG_(n). Further, a parent character bone group of the character bone group CBG_(n) is referred to as CBG_(n−1).

L_(Gn) means a local matrix of the character bone group CBG_(n) that expresses a position (trans) and rotation (rotate) of the end point of the character bone group CBG_(n) in the CBG_(n−1) coordinate system. When it is assumed that the bones constituting the character bone group CBG_(n) are CB_(m″), . . . , CB_(m−1), CB_(m), from the parent side, the following equation can be obtained. L_(Gn)=L_(m)L_(m−1) . . . L_(m″)

“M_(Gn)” means an object matrix of the character bone group CBG_(n) that expresses a position (trans) and rotation (rotate) of the end point of the character bone group CBG_(n) in the object coordinate system (=node 0 origin). M_(Gn)=L_(Gn)L_(Gn−1) . . . L_(G1)=L_(Gn)M_(Gn−1)=L_(m)L_(m−1) . . . L_(m″)M_(m″−1)=M_(m)

“L_(Gn) ^((ini))” means the local matrix L_(Gn) of the character bone group CBG_(n) at the initial pose of the character 130. L_(Gn) ^((ini))=L_(m) ^((ini))L_(m−1) ^((ini)) . . . L_(m″) ^((ini))

“M_(Gn) ^((ini))” means the object matrix M_(Gn) of the character bone group CBG_(n) at the initial pose of the character 130.

$\begin{matrix} {M_{Gn}^{({ini})} = {L_{Gn}^{({ini})}L_{{Gn} - 1}^{({ini})}\mspace{14mu}\ldots\mspace{14mu} L_{G\; 1}^{({ini})}}} \\ {= {L_{Gn}^{({ini})}M_{{Gn} - 1}^{({ini})}}} \\ {= {L_{m}^{({ini})}L_{m - 1}^{({ini})}\mspace{14mu}\ldots\mspace{14mu} L_{m^{''}}^{({ini})}M_{m^{''} - 1}^{({ini})}}} \\ {= M_{m}^{({ini})}} \end{matrix}$

Similarly, it is assumed that an n-th bone group of the template 120 is referred to as a template bone group TBG_(n). Further, a parent template bone group of the template bone group TBG_(n) is referred to as TBG_(n−1).

“L_(G′n)” means a local matrix of the template bone group TBG_(n) that expresses a position (trans) and rotation (rotate) of the end point of the template bone group TBG_(n) in the TBG_(n−1) coordinate system. When it is assumed that the bones constituting the template bone group TBG_(n) are TB_(k″), . . . , TB_(k−1), TB_(k), from the parent side, the following equation can be obtained. L_(G′n)=L′_(k)L′_(k−1) . . . L′_(k″)

“M_(G′n)” means an object matrix of the template bone group TBG_(n) that expresses a position (trans) and rotation (rotate) of the end point of the template bone group TBG_(n) in the object coordinate system (=node 0 origin).

$\begin{matrix} {M_{G^{\prime}n} = {L_{G^{\prime}n}L_{{G^{\prime}n} - 1}\mspace{14mu}\ldots\mspace{14mu} L_{G^{\prime}1}}} \\ {= {L_{G^{\prime}n}M_{{G^{\prime}n} - 1}}} \\ {= {L_{k}^{\prime}L_{k - 1}^{\prime}\mspace{14mu}\ldots\mspace{14mu} L_{k^{''}}^{\prime}M_{k^{''} - 1}^{\prime}}} \\ {= M_{k}^{\prime}} \end{matrix}$

“L_(G′n) ^((ini))” means the local matrix L_(G′n) of the template bone group TBG_(n) at the initial pose of the template 120. L_(G′n) ^((ini))=L′_(k) ^((ini))L′_(k−1) ^((ini)) . . . L′_(k″) ^((ini))

“M_(G′n) ^((ini))” means the object matrix M_(G′n) of the template bone group TBG_(n) at the initial pose of the template 120.

$\begin{matrix} {M_{G^{\prime}n}^{({ini})} = {L_{G^{\prime}n}^{({ini})}L_{{G^{\prime}n} - 1}^{({ini})}\mspace{14mu}\ldots\mspace{14mu} L_{G^{\prime}1}^{({ini})}}} \\ {= {L_{G^{\prime}n}^{({ini})}M_{{G^{\prime}n} - 1}^{({ini})}}} \\ {= {L_{k}^{\prime{({ini})}}L_{k - 1}^{\prime{({ini})}}\mspace{14mu}\ldots\mspace{14mu} L_{k^{''}}^{\prime{({ini})}}M_{k^{''} - 1}^{\prime{({ini})}}}} \\ {= M_{k}^{\prime{({ini})}}} \end{matrix}$

FIG. 9 is a view showing two corresponding bone groups of the template 120 and the character 130.

1.3. Copy of Pose

In an operation of copying the pose, information about an initial pose of the template 120 and an initial pose of the character 130 are provided first. Here, the initial poses of the template 120 and the character 130 are not necessarily the same.

Thereafter, the poses (current poses) of the bones of the template 120 are successively provided so that the statuses (matrix of each bone) of the character 130 having the same poses are to be determined.

First, a direction of the bone group of the character 130 is obtained based on the direction of the bone group of the template 120. Subsequently, directions of the bones in the bone groups are obtained.

1.4. Coordinate System Adjusting Matrix

The coordinate systems of the bone groups of the template 120 and the character 130 are different. Thus, as described above, first, a “coordinate system adjusting matrix” that expresses a relationship between the coordinate systems of the bone groups for the template 120 and the character 130 is calculated based on information about the character bone group and the template bone group at an initial state. This operation is performed once at first. This operation is explained. Here, the coordinate system adjusting matrix is a rotation matrix which does not change the length of the bone (trans unit is not specifically used).

(1.4.1. Generating Overlapped State)

It is assumed that a directed line segment from a starting point to an end point of a bone group is a “bone group axis”. It is necessary to have directions of the bone group axes of the template 120 and the character 130 become the same in order to calculate the coordinate system adjusting matrix. Thus, an “overlapped state” is generated as follows.

Using the template 120 and the character 130 at an initial state, the pose of the character 130 is determined to match the pose of the template 120.

First, the bone groups CBG_(n) of the character 130 at the initial state are rotated (by a simple rotation, which will be explained later) such that each of the directions of the axes of the bone groups CBG_(n) becomes the same as the direction of the axis of the corresponding bone group TBG_(n) of the template 120 at the initial state. With this operation, the character 130 comes to have the same pose as that of the template 120 at the initial state. This state of the character is referred to as an “overlapped state”. The local matrix and the object matrix of the bone group are referred to as L_(Gn) ^((ovl)) and M_(Gn) ^((ovl)), respectively.

The overlapped state can be recursively obtained as follows. Here, it is assumed that L_(Gn−1) ^((ovl)) and M_(Gn−1) ^((ovl)) of the (n−1)th bone group are obtained (the directions of the axes of CBG_(n) and TBG_(n) coincide).

Here, a vector “vec” of the bone group axis of the character bone group CBG_(n+1) of the character 130 in the coordinate system of the character bone group CBG_(n) can be expressed as follows. vector “vec”=trans unit of L_(Gn+1) ^((ini))   (101)

Similarly, a vector “vecToTemplate” of the bone group axis of the template bone group TBG_(n+1) of the template 120 in the coordinate system of the character bone group CBG_(n) can be expressed as follows. vector “vecToTemplate” =trans unit of(M _(G)′_(n+1) ^((ini)) −M _(G)′_(n) ^((ini)))(L _(Gn) ^((ini)) M _(Gn−1) ^((ovl)))⁻¹   (102)

Here, the “simple rotation” is defined as follows.

A rotation that satisfies the following conditions is referred to as a simple rotation from unit vectors v_(a) to v_(b) where the unit vectors v_(a) and v_(b) have a common starting point.

-   -   The rotation is around an axis which is perpendicular to a plane         formed by the unit vectors v_(a) and v_(b).     -   The unit vector v_(a) overlaps the unit vector v_(b) by the         rotation.

Here, if the directions of the unit vectors v_(a) and v_(b) are the same, no rotation is performed (expressed by a unit matrix as the rotation matrix).

If the directions of the unit vectors v_(a) and v_(b) are opposite from each other, it is impossible to define.

(Rotation Matrix H_(n))

As shown in FIG. 10, a rotation matrix H_(n) to generate an overlapped state of the character bone group CBG_(n+1) is calculated as follows. H_(n) means a simple rotation from “vec” to “vecToTemplate”  (103)

H_(n) is a rotation matrix in the coordinate system of the character bone group CBG_(n).

By using the rotation matrix H_(n), the local matrix L_(Gn) ^((ovl)) of the character bone group CBG_(n) at the overlapped state can be obtained as follows. L _(Gn) ^((ovl)) =H _(n)(rotation unit of L _(Gn) ^((ini)))(trans unit of L _(Gn) ^((ini)))=H _(n) L _(Gn) ^((ini))  (104)

Further, the object matrix M_(Gn) ^((ovl)) of the character bone group CBG_(n) at the overlapped state can be obtained as follows. M_(G1) ^((ovl))=L_(G1) ^((ovl))  (105) M_(Gn) ^((ovl))=L_(Gn) ^((ovl))L_(Gn−1) ^((ovl)) . . . L_(G1) ^((ovl))  (106)

With this operation, the local matrix L_(Gn) ^((ovl)) of the character bone group CBG_(n) at the overlapped state, and the object matrix M_(Gn) ^((ovl)) of the character bone group CBG_(n) at the overlapped state are obtained.

(1.4.2. Calculating Coordinate System Adjusting Matrix R_(n))

FIG. 11 is a view showing two coordinate systems of the template 120 and the character 130.

By the operation explained above with reference to FIG. 10, the directions of the bone groups of the character 130 at the overlapped state L_(Gn) ^((ovl)), M_(Gn) ^((ovl)) and the template 120 at the initial state L_(G)′_(n) ^((ini)), M_(G)′_(n) ^((ini)) become the same. However, the coordinate systems of the bone groups are different. Thus, a “coordinate system adjusting matrix” R_(n) which can adjust the difference in the coordinate systems is calculated. In this operation, the matrices in the equations do not include the trans unit. M_(G)′_(n) ^((ini))=R_(n)M_(Gn) ^((ovl))  (107)

The left side of the equation (107) can be expressed as follows by breaking down. M_(G)′_(n) ^((ini))=R_(n)L_(Gn) ^((ovl))M_(Gn−1) ^((ovl))  (108)

Further, when inputting “n−1” in “n” of the equation (107) to obtain M_(Gn−1) ^((ovl)), the next equation is obtained. M _(Gn−1) ^((ovl))=(R _(n−1))⁻¹ M _(G)′_(n−1) ^((ini))  (109)

From the equations (108) and (109), the next equation can be obtained. M _(G)′_(n) ^((ini)) =R _(n) L _(Gn) ^((ovl))(R _(n−1))⁻¹ M _(G)′_(n−1) ^((ini))  (110)

On the other hand, as M_(G′n) ^((ini)) can be expressed as M_(G)′_(n) ^((ini))=L_(G)′_(n) ^((ovl))M_(G)′_(n−1) ^((ini)), the next equation is obtained. L _(G)′_(n) ^((ini)) =R _(n) L _(Gn) ^((ovl))(R _(n−1))⁻¹  (111)

With this, R_(n) can be recursively obtained as follows. R ₁=(rotation unit of L _(G)′_(n) ^((ini)))(L _(G1) ^((ovl)))⁻¹   (112) R _(n)=(rotation unit of L _(G)′_(n) ^((ini)) R _(n−1))(L _(Gn) ^((ovl)))⁻¹   (113) 1.5. Determining Character State

By the operation explained above with reference to FIG. 11, the coordinate system adjusting matrix R_(n) that expresses a relationship between the template 120 and the character 130 is obtained.

Thus, the state of the character 130 is determined to take the same pose as the pose (current state) (rotate, trans) of the template 120 which is successively obtained.

First, the state of the bone group is calculated, and then, the state of the bone in the bone group is obtained.

(1.5.1. Determining Current State of Character Bone Group CBG_(n))

The current state of the character 130 is determined for each of the bone groups. At this time, the coordinate system adjusting matrix R_(n) is used.

The current local matrix of the character bone group CBG_(n) can be determined as follows using the current local matrix of the terminal bone group TBG_(n). Rotation of L_(Gn) =(R _(n))⁻¹(rotation of L _(G)′_(n))R _(n−1)  (114)

With this, the current state of the character bone group CBG_(n) is determined.

(1.5.2. Determining Current State of Character Bone in Character Bone Group CBG_(n))

Subsequently, the current state of the character bone in the character bone group CBG_(n) is determined.

FIG. 12 is a view showing states of bones in the bone groups.

In this process, the ratio of the length of each of the bones is used so that a coefficient d_(m) for position conversion of the character bones and the template bones in the bone groups at the initial state and accumulated lengths are calculated.

As shown in FIG. 12, a length of a character bone CB_(m) which is normalized such that a total length of the bones in the bone group becomes 1 is referred to as l_(m). Similarly, a length of a template bone TB_(k) which is normalized such that a total length of the bones in the bone group becomes 1 is referred to as l′_(k).

Further, in order to simply calculate the accumulated length of the character bones from the accumulated length of the template bones, a coefficient (ratio) d_(m) for the fraction of lengths is obtained as follows. Σ_((i=1˜m))l_(i)=Σ_((i=1˜k−1))l′_(i)+d_(m)*l_(k)  (115)

FIG. 13 is a view showing an example of the current character bones in the bone group coordinate system.

As shown in FIG. 13, using the coefficient (ratio) d_(m), the total length of the current template bones is divided by a ratio of the length of the character bone. With this operation, the current position of the character bone in the bone group coordinate system can be calculated.

A ratio of the total length of the template bones in the bone group and the total length of the character bones in the corresponding bone group is referred to as a ratio. L″_(k) is defined as a multiple obtained by multiplying the ratio by the trans unit of L′_(k).

As described above in the equation (114), the following relation exists for the Rotation of L_(Gn) =(R _(n))⁻¹(rotation of L _(G)′_(n))R _(n−1)  (116)

Thus, L_(Gn) can be expressed as follows. L _(Gn)=(R _(n))⁻¹ L″ _(k) L″ _(k−1) . . . R _(n−1)  (117)

First, L′″_(k) is defined as follows in order to first determine the end point of the character bone. L′″_(k)=L″_(k)L″_(k−1) . . . R_(n−1) L′″_(k−1)=L″_(k−1) . . . R_(n−1)  (118)

At this time, the current end point position Pnt_(m) of the character bone CB_(m) in the bone group coordinate system becomes as follows by interpolating a trans between L′″_(k−1) and L′″_(k) by d_(m). Pnt_(m)=trans of((1−d _(m))*L′″ _(k−1) +d _(m) *L′″ _(k))   (119)

FIG. 13 shows the operation of determining the character bone CB_(m).

FIG. 14 is a view for explaining an operation of determining a pose of the character.

With the above process, the direction of the axis of the character bone CB_(m) is determined. However, “torsion” around the axis is not determined. Thus, the character bone CB_(m) is determined using a simple rotation Q_(m) from the axis direction (initVec) at the initial state to the determined axis direction (currentVec) as shown in FIG. 13.

When the axis directions are expressed in the CB_(m−1) coordinate system, these become as follows. vector initVec:(trans of L_(m+1) ^((ini)))(rotation of L_(m) ^((ini)))  (120) vector currentVec:trans of(Pnt_(m+1)−Pnt_(m))(W_(m′−1))⁻¹  (121)

Here, W_(m′) is an integrating matrix in the bone group as follows. W_(m′)=L_(m′)L_(m′−1) . . . L_(m)  (122)

As W_(m′)=L_(m′)W_(m′−1), these values can be recursively obtained from L_(m).

However, only for the last bone L_(m′) in the bone group, it is obtained by a backcalculation from L_(Gn) so that L_(Gn)=L_(m′)L_(m′−1) . . . L_(m) is complied with. For L₁,rotation is L₁ ^((ini))Q₁ and trans is Pnt₁  (123) W₁=L₁  (124) For L_(m),rotation is L_(m) ^((ini))Q_(n) and trans is Pnt_(m)(W_(m″−1))⁻¹  (125) W_(m′)=L_(m′)W_(m′−1)  (126)

Here, when m=m′, the end point bone can be expressed as follows. L _(m′) =L _(Gn)(W _(m′−1))⁻¹  (127)

As described above, the local matrix of each of the character bones can be obtained and the pose of the character 130 is determined.

2. Second Example in Detail

2.1. Structure of Skeleton Model

(2.1.1. Node, Bone)

Parts of the skeleton model are connected in a parent-child relationship. Each of the parts is referred to as a “node”. A “bone” is a node. The origin of the node is referred to as a starting point of the bone, and the origin of one of the child nodes is referred to as an end point of the bone. A directed line segment from the starting point to the end point is referred to as a “bone axis”. A vector that expresses the bone axis is referred to as a “bone axis vector”, and a normalized result of the bone axis vector to be a magnitude of 1 is referred to as a “bone axis unit vector”. The magnitude of the bone axis vector is a length of the bone.

(2.1.2. Template and Character)

When copying a pose of a skeleton model to another skeleton model, the skeleton model from which the pose is copied is referred to as a “template”, the skeleton model to which the pose is copied is referred to as a “character”. Generally, the number of nodes included in the template and the number of nodes included in the character are different. Further, there are no particular rules in directions of the coordinate systems of the nodes, respectively, so that the directions of the coordinate systems are different from each other.

(2.1.3. Bone Group)

A “bone group” means an assembly of linearly connected one or more bones. Here, it is necessary that all of the adjacent bones have a relationship “parent to child” (or all of the adjacent bones have a relationship “child to parent”) when checking from one end of the bone group to another end of the bone group. Among both ends of the bone group, a parent side end becomes a starting point, and a child side end becomes an end point. More accurately, the starting point of the bone at the most parent side of the bone group becomes a starting point of the bone group. Similarly, the end point of the bone at the most child side of the bone group becomes the end point of the bone group. A directed line segment from the starting point toward the end point of the bone group is referred to as a “bone group axis”. A vector that expresses the bone group axis is referred to as a “bone group axis vector”, and a normalized result of the bone group axis vector to be a magnitude of 1 is referred to as a “bone group axis vector unit”.

The bones included in the skeleton model are divided into plural of the bone groups. Here, each of the bones does not belong to more than one bone group. The parent-child relationship of the bone groups is similar to the parent-child relationship of the bones.

Prior to copying the pose of the template to the character, the bone groups are formed for both the template and the character, and the bone groups which are at the similar part of the template and the character are in correspondence with each other in 1 to 1 relationship.

When copying the pose, first, the direction of the bone group of the character is determined to coincide based on the direction of the corresponding bone group of the template. Then, the directions of the bones in the bone group are determined.

2.2. Coordinate System

(2.2.1. Coordinate System of Bone)

Each of the nodes has a coordinate system. The coordinate system of a bone means a coordinate system of the node which corresponds to the bone. As described above, there are no particular rules in directions of the coordinate systems of the nodes (bones). Thus, the vector which expresses the direction of the bone axis in the coordinate system of the bone becomes different for each of the bones. For example, the bone axis may be in an x-axis direction of the bone for one bone, while the bone axis may be in a y-axis direction of the bone for another bone.

(2.2.2. Coordinate System of Bone Group)

It is defined that a coordinate system of a bone group is a multiple obtained by multiplying a coordinate system of a first bone which is at the most parent side of the bone group with “a simple rotation from a first bone axis to the bone group axis”. For the simple rotation, see a following Appendix A.

(2.2.3. Object Coordinate System)

Although there are no particular rules in the direction of the coordinate system of the bone, there is an exception for the bone at the most parent side. The node at the most parent side in each of the skeleton models is referred to as a “root node”. In this example, it is assumed that directions of the coordinate systems of the root nodes are the same for all of the skeleton models. For example, a front of each of the skeleton models becomes the z-axis and a top of each of the skeleton models becomes the y-axis. The coordinate system of the root node (root bone) is referred to as an “object coordinate system”.

(2.2.4. Object Matrix, Local Matrix)

In order to express the direction of a bone (posture or rotation) by a matrix, a reference coordinate system is also necessary in addition to the coordinate system of the respective bone. The matrix represented by using the object coordinate system as a standard coordinate system is referred to as an object matrix of the bone. The matrix represented by using the parent coordinate system as the standard coordinate system is referred to as a local matrix of the bone. The same can be referred to for the bone groups.

2.3. Symbol, Basic Equations

Symbols are defined as follows.

TBG_(n): n-th template bone group.

CBG_(n): n-th character bone group.

BG_(n): n-th bone group pair composed of TBG_(n) and CBG_(n).

M_(Gn)′: object matrix of TBG_(n).

L_(Gn)′: local matrix of TBG_(n).

M_(Gn): object matrix of CBG_(n).

L_(Gn): local matrix of CBG_(n).

TB_(k): k-th template bone.

M_(k)′: object matrix of TB_(k).

L_(k)′: local matrix of TB_(k).

CB_(m): m-th character bone.

M_(m): object matrix of CB_(m).

L_(m): local matrix of CB_(m).

As described above, “′” means the template, and without “′” means the character. Further, “^((ini))” means an initial state (initial pose) of a bone, without “^((ini))” means a current state (current pose) of a bone. Unless otherwise noted, the number (index) increases one by one from the parent to the child. It is assumed that the matrix size is three rows by three columns, and for the alignment of the components, a row vector is multiplied by a matrix from right side.

There is a following relationship between the object matrix and the local matrix.

$\begin{matrix} \begin{matrix} {M_{Gn} = {L_{Gn}L_{{Gn} - 1}L_{{Gn} - 2}}} \\ {= {L_{Gn}{M_{{Gn} - 1}(202)}}} \end{matrix} & (201) \end{matrix}$

This same relationship can be applied for the template bone group. Similarly, the same relationship can be applied for the object matrix and the local matrix of the bone.

2.4. Copying Pose

(2.4.1. Initial State)

In this example, it is assumed that initial states of the template and the character are previously provided where the template and the character take the same pose at the initial state. Here, the “same pose” does not mean that the bone (or the bone group) of the template and the character are directed in the same direction.

FIG. 15 is a view showing an example of a template (standard skeleton model 124) and a character (target skeleton model 134) at initial states. Although both of the template and the character take the same pose, directions of collarbones 124 a and 134 a, respectively, of them are different, as can be understood from FIG. 15.

(2.4.2. Copying Pose)

“Copying pose” means to determine a pose of the target skeleton model 134 to be the same as a pose (current pose) of the standard skeleton model 124 when the current pose of the standard skeleton model 124 is provided.

In this example, the object matrices M_(k′) of all of the bones of the standard skeleton model 124 are provided. Then, the object matrices M_(m) of all of the bones of the target skeleton model 134 are to be determined.

When copying the pose, first, the direction of each of the character bone groups of the target skeleton model 134 is determined based on the direction of the corresponding template bone group of the standard skeleton model 124. Subsequently, the directions of the character bones in the character bone group are determined.

(2.4.3. Determining Direction)

The coordinate system is different for each of the bone groups, similar in the fact that the coordinate system is different for each of the bones. Thus, a coordinate system adjusting matrix for converting the coordinate system of the template bone group to the coordinate system of the corresponding character bone group is obtained.

When determining the direction of the character bone group based on the current direction of the template bone group, the coordinate system adjusting matrix is used. In addition, it is necessary to take the difference between the directions of the template bone group and the character bone group at the initial pose into consideration.

2.5. Coordinate System Adjusting Matrix

First, the coordinate system adjusting matrix is obtained. It is necessary to perform this process only once when the initial poses of the template and the character are provided.

(2.5.1. Correcting Initial State)

FIG. 16 is a view showing a simple rotation for coinciding axes. In FIG. 16, (a) shows a template bone group TBG_(n) 300 and (b) shows a character bone group CBG_(n) 350.

In (a) of FIG. 16, “302” expresses the template bone group TBG_(n) 300 at the initial state, and “310” expresses the template bone group TBG_(n−1) at the initial state. Similarly, in (b) of FIG. 16, “352” expresses the character bone group CBG_(n) 300 at the initial state, and “360” expresses the character bone group CBG_(n−1) at the initial state.

In (a) of FIG. 16, a simple rotation is performed on the template bone group TBG_(n) 300 at the initial state 302 in order to coincide the axis direction of the template bone group TBG_(n) 300 with that of the character bone group CBG_(n) 350 at the initial state (1). This process is referred to as a simple rotation for coinciding axes of a bone group BG_(n).

After having applied the simple rotation for coinciding axes of the bone group BG_(n), the template bone group TBG_(n) 300 takes a modified initial state 304. A matrix L_(Gn)′^((mod)) expressing a direction of the template bone group TBG_(n) 300 at the modified initial state 304 in the TBG_(n−1) coordinate system is obtained as follows.

Here, a matrix expressing the simple rotation for coinciding axes of the bone group BG_(n) in the object coordinate system is referred to as S_(Gn)*, and a matrix expressing the simple rotation for coinciding axes of the bone group BG_(n) in the initial TBG_(n) coordinate system is referred to as S_(Gn). At this time, S_(Gn) can be expressed as follows. S_(Gn)=M_(Gn)′^((ini))S_(Gn)*M_(Gn)′^((ini)−1)  (203)

Further, a direction of the template bone group TBG_(n) at the modified initial state 304 in the initial TBG_(n−1) coordinate system is referred to as L_(Gn)′^((mod)). At this time, L_(Gn)′^((mod)) can be expressed as follows.

$\begin{matrix} {L_{Gn}^{\prime{({mod})}} = {S_{Gn}L_{Gn}^{\prime{({ini})}}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(204)} \\ {= {M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{Gn}^{{\prime{({ini})}} - 1}L_{Gn}^{\prime{({ini})}}}} & {(205)} \\ {= {M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}}} & {(206)} \end{matrix}$

Further, L_(Gn)′^((mod)) can be obtained as follows as well.

Here, a matrix expressing the simple rotation for coinciding axes of the bone group BG_(n) in the initial TBG_(n−1) coordinate system is referred to as S_(Gn−1). At this time, S_(Gn−1) can be expressed as follows. SG_(Gn−1)=M_(Gn−1)′^((ini))S_(Gn)*M_(Gn−1)′^((ini)−1)  (207)

Then, L_(Gn′) ^((mod)) can be expressed as follows.

$\begin{matrix} {L_{Gn}^{\prime{({mod})}} = {L_{Gn}^{\prime{({ini})}}S_{{Gn} - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(208)} \\ {= {L_{Gn}^{\prime{({ini})}}M_{{Gn} - 1}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}}} & {(209)} \\ {= {M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}}} & {(210)} \end{matrix}$ (2.5.2. Calculating Coordinate System Adjusting Matrix)

Then, a matrix for adjusting the difference in the coordinate axes of the template bone group TBG_(n) 300 at the modified initial state 304 and the character bone group CBG_(n) 350 at the initial state 352 is obtained. The direction of the template bone group TBG_(n) 300 at the modified initial state 304 in the object coordinate system is expressed as L_(Gn)′^((mod))M_(Gn−1)′^((ini)), and the direction of the character bone group CBG_(n) 350 at the initial state 352 in the object coordinate system is expressed as M_(Gn) ^((ini)). A matrix A_(Gn) expressing the conversion between them in the initial modified coordinate system TBG_(n) is the coordinate system adjusting matrix to be obtained. A_(Gn) can be obtained as follows.

$\begin{matrix} {A_{Gn} = {M_{Gn}^{({ini})}\left( {L_{Gn}^{\prime{({mod})}}M_{{Gn} - 1}^{\prime{({ini})}}} \right)}^{- 1}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(211)} \\ {= {M_{Gn}^{({ini})}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}L_{Gn}^{{\prime{({mod})}} - 1}}} & {(212)} \\ {= {M_{Gn}^{({ini})}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}M_{{Gn} - 1}^{\prime{({ini})}}S_{Gn}^{*{- 1}}M_{Gn}^{{\prime{({ini})}} - 1}}} & {(213)} \\ {= {M_{Gn}^{({ini})}S_{Gn}^{*{- 1}}M_{Gn}^{{\prime{({ini})}} - 1}}} & {(214)} \end{matrix}$

A_(Gn) can be obtained as follows.

First, the coordinate system adjusting matrix A_(Gn)* in the object coordinate system is obtained in the next equation. L_(Gn)′^((mod))M_(Gn−1)′^((ini))A_(Gn)*=M_(Gn) ^((ini))  (215) Thus, A_(Gn)*=M_(Gn−1)′^((ini)−1)L_(Gn)′^((mod)−1)M_(Gn) ^((ini))  (216)

Then, the coordinate system adjusting matrix A_(Gn) is obtained by converting A_(Gn)* to be expressed in the initial modified coordinate system TBG_(n).

$\begin{matrix} {A_{Gn} = {\left( {L_{Gn}^{\prime{({mod})}}M_{{Gn} - 1}^{\prime{({ini})}}} \right){A_{Gn}^{*}\left( {L_{Gn}^{\prime{({mod})}}M_{{Gn} - 1}^{\prime{({ini})}}} \right)}^{- 1}}} & {{~~~}(217)} \\ {= {L_{Gn}^{\prime{({mod})}}M_{{Gn} - 1}^{\prime{({ini})}}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}L_{Gn}^{{\prime{({mod})}} - 1}M_{Gn}^{({ini})}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}L_{Gn}^{{\prime{({mod})}} - 1}}} & {(218)} \\ {= {M_{Gn}^{({ini})}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}L_{Gn}^{{\prime{({mod})}} - 1}}} & {(219)} \\ {= {M_{Gn}^{({ini})}S_{Gn}^{*{- 1}}M_{Gn}^{{\prime{({ini})}} - 1}}} & {(220)} \end{matrix}$ 2.6. Determining Direction of Bone Group

Then, the direction (posture or rotation) of the character bone group CBG_(n) 350 at a current state 306 is obtained in order to determine the pose of the character from the provided current pose of the template (2).

In this operation, the entirety of the template at the current state is moved such that the direction of the current template bone group TBG_(n−1) coincides the direction of the initial template bone group TBG_(n−1) at the initial state 302, first. Then, under this state, the directions of the template bone group TBG_(n) and the character bone group CBG_(n) are compared.

A matrix expressing the rotation of the template bone group TBG_(n) 300 at the initial state 302 to the current state 306 in the initial TBG_(n) coordinate system is referred to as D_(n). D_(n) can be obtained from the next equation. D_(n)=L_(Gn)′L_(Gn)′^((ini)−1)  (221)

Then, D_(n) is divided into a bending matrix D_(Bn) and a torsion matrix D_(Tn). D_(n)=D_(Tn)D_(Bn)  (222)

For the bending matrix D_(Bn), a scale is multiplied to a bending amount in accordance with a necessity to obtain the matrix D_(Bn) ^((adj)). Then, the adjusted matrix D_(Bn) ^((adj)) is obtained by multiplying D_(Bn) ^((adj)) with D_(Tn). This operation is explained in detail later. D_(n) ^((adj))=D_(Tn)D_(Bn) ^((adj))  (223)

Then, the direction for the local matrix L_(Gn(1))′ is obtained by multiplying the adjusted matrix D_(n) ^((adj)) by the template bone group TBG_(n) at the initial modified state 304 (3).

$\begin{matrix} {L_{{Gn}{(1)}}^{\prime} = {{D_{n}}^{({adj})}L_{Gn}^{\prime{({mod})}}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(224)} \\ {= {D_{n}^{({adj})}M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}}} & {{~~~~~~}(225)} \end{matrix}$

Further, by multiplying M_(Gn−1)′ by the direction for the local matrix L_(Gn(1))′, the direction for the character expressed in the object coordinate system is obtained.

$\begin{matrix} {M_{{Gn}{(1)}}^{\prime} = {L_{{Gn}{(1)}}^{\prime}M_{{Gn} - 1}^{\prime}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(226)} \\ {= {D_{n}^{({adj})}M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}M_{{Gn} - 1}^{\prime}}} & {{~~~~~~}(227)} \end{matrix}$

Subsequently, by multiplying the coordinate system adjusting matrix A_(Gn) to the direction for the matrix M_(Gn(1))′ in order to adjust the differences in the coordinate systems of TBG_(n) and CBG_(n), a matrix M_(Gn) for the character bone group CBG_(n) 350 at the current state is obtained.

$\begin{matrix} {M_{Gn} = {A_{Gn}M_{{Gn}{(1)}}^{\prime}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(228)} \\ {= {A_{Gn}D_{n}^{({adj})}M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}M_{{Gn} - 1}^{\prime}}} & {{~~~~~~}(229)} \end{matrix}$

As “M_(Gn)′^((ini))S_(Gn)*M_(Gn−1)′^((ini)−1)” at the right side is obtained only from the initial pose, it is only necessary to calculate once at first.

If the bending amount is not adjusted, the matrix M_(Gn) becomes as follows.

$\begin{matrix} {M_{Gn} = {A_{Gn}L_{Gn}^{\prime}L_{Gn}^{{\prime{({ini})}} - 1}M_{Gn}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}M_{{Gn} - 1}^{\prime}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(230)} \\ {= {A_{Gn}L_{Gn}^{\prime}M_{{Gn} - 1}^{\prime{({ini})}}S_{Gn}^{*}M_{{Gn} - 1}^{{\prime{({ini})}} - 1}M_{{Gn} - 1}^{\prime}}} & {{~~~~~~}(231)} \end{matrix}$ 2.7. Division of D_(n) into Bending Matrix D_(Bn) and Torsion Matrix D_(Tn), and Adjustment of Bending Amount (2.7.1. Adjustment of Bending Amount)

When copying the pose of the template into the character, there may be a case where a priority is given to relative positional relationships between both ends of the bones (bone groups) rather than the directions of the bones (bone groups). In such a case, an adjustment depending on the lengths of the bones (bone groups) is necessary. For example, when the distance between ends of the character bone group CBG_(n) is longer than the distance between ends of the template bone group TBG_(n), and the character bone group CBG_(n) is also rotated for an angle the same as that of the template bone group TBG_(n), the relative movement amount of the ends becomes larger for the character bone group CBG_(n). In order to adjust this difference, it is necessary to reduce just a bending component of the character bone group CBG_(n) while maintaining the amount of the torsion component among the rotations.

(2.7.2. Division of D_(n) into Bending Matrix D_(Bn) and Torsion Matrix D_(Tn))

As described above, the matrix expressing the rotation of the template bone group TBG_(n) 300 at the initial state 302 to the current state 306 in the initial TBG_(n) coordinate system is referred to as D_(n). D_(n) can be determined as the product of the bending matrix D_(Bn) and the torsion matrix D_(Tn) as shown in the next equation. D_(n)=D_(Tn)D_(Bn)  (232)

It is assumed that a vector that expresses the TBG_(n) axis at the current state in the object coordinate system is v_(Gn), and a vector obtained by converting v_(Gn) in the initial TBG_(n) coordinate system is v_(Gn(2)). The equation for obtaining v_(Gn(2)) from v_(Gn) is as follows. V_(Gn(2))=v_(Gn)M_(Gn−1)′L_(Gn)′^((ini)−1)  (233)

On the other hand, a vector that expresses the TBG_(n) axis in the own TBG_(n) coordinate system is referred to as v_(Gn) ^((self)). The above described bending matrix D_(Bn) can be obtained as a simple rotation from v_(Gn) ^((self)) to v_(Gn(2)). With this bending matrix D_(Bn), the torsion matrix D_(Tn) can be obtained from the next equation. D_(Tn)=D_(n)D_(Bn) ⁻¹  (234) (2.7.3. Adjusting Bending Amount and Composing Bending Component and Torsion Component)

Here, a ratio of a distance between ends of the template bone group TBG_(n) to a distance between ends of the character bone group CBG_(n) is referred to as r_(n). A matrix that magnifies r_(n) to the bending angle of the bending matrix D_(Bn) is referred to as D_(Bn) ^((adj)). With this, the bending component and the torsion component can be composed to obtain an adjusted matrix D_(n) ^((adj)) as shown in the next equation. D_(n) ^((adj))=D_(Tn)D_(Bn) ^((adj))  (235) 2.8. Determining Direction of Bone in Bone Group

An arrangement of the character bone in the character bone group CBG is obtained from the arrangement of the template bone in the corresponding template bone group TBG (here, the index is omitted).

(2.8.1. Common Coordinate System)

As described above, the TBG coordinate system can be obtained by multiplying a simple rotation from the TB₁ axis by the TBG axis, in the TB₁ coordinate system. Here, a new “common coordinate system” of a bone TB_(k) (k=1, 2, . . . ) (which will be referred to as a “common TB_(k) coordinate system”) is determined as follows.

It is assumed that the common TB_(k) coordinate system can be obtained by multiplying a simple rotation from the TBG axis to the TB_(k) axis, from the TBG coordinate system for the template at the initial state. The common TB₁ coordinate system becomes the same as the original TB₁ coordinate system.

Similar to the template bone, the common coordinate system for the character bone is determined. At this time, an adjustment by the coordinate system adjusting matrix A_(G) between the bone groups is applied. It is assumed that a common CB_(m) coordinate system can be obtained by multiplying a simple rotation from the CBG axis to the CB_(m) axis, of the CBG coordinate system adjusted by A_(G) for the character at the initial state.

The common bone coordinate system has the following characteristic. The directions of the coordinate systems of the bones are different for each of the bones. Thus, the value of the axis vector unit that expresses the axis of the TB_(k) in the TB_(k) coordinate system is different for each of the bones. On the other hand, the value of the axis vector unit that expresses the axis of the TB_(k) in the common TB_(k) coordinate system becomes the same for each of the bones in the bone group.

Similarly, when the torsion rotation (rotation around the bone axis) of the bone TB_(k) is expressed in the TB_(k) coordinate system, the matrix becomes different for each of the bones. However, when the torsion rotation (rotation around the bone axis) of the bone TB_(k) is expressed in the common TB_(k) coordinate system, the matrix becomes the same. These are similar for the character bones. This characteristic can be used when interpolating the directions of bones (posture or rotation).

It is assumed that a matrix expressing the direction of the bone in the common coordinate system in the object coordinate system is Ω_(j). Further, it is assumed that a matrix expressing the direction of the bone in the common coordinate system in the bone group coordinate system is Λ_(j). Here, “j” is a number of a bone. In the following, “′” means the template, without “′” means the character. “^((ini))” means an initial state (initial pose) of a bone.

(2.8.1.1. Common Coordinate System of Template Bone)

A conversion matrix between the original coordinate system of the bone and the common coordinate system is obtained using the initial state of the template.

It is assumed that a vector expressing the TBG axis in the object coordinate system is v_(G)′^((ini)). It is assumed that a vector expressing the TB_(k) axis in the object coordinate system is v_(k)′^((ini)). It is assumed that a simple rotation from v_(G′) ^((ini)) to V_(k)′^((ini)) is S_(k)′.

It is assumed that a matrix expressing the direction of the common TB_(k) coordinate system in the object coordinate system is Ω_(k)′^((ini)). As the matrix expressing the direction of the TBG in the object coordinate system is M_(G)′^((ini)), Ω_(k)′^((ini)) can be obtained as follows by multiplying S_(k)′ by M_(G)′^((ini)). Ω_(k′) ^((ini))=M_(G)′^((ini))S_(k)′  (236)

Here, a matrix expressing the direction of the original TB_(k) coordinate system in the object coordinate system is M_(k)′^((ini)). Thus, a conversion matrix between Ω_(k)′^((ini)) and M_(k)′^((ini)) is previously obtained. Specifically, a matrix expressing a conversion from M_(k)′^((ini)) to Ω_(k)′^((ini)) in the original TB_(k) coordinate system is Φ_(k)′ as is obtained in the next equation.

$\begin{matrix} {\Phi_{k}^{\prime} = {\Omega_{k}^{\prime{({ini})}}M_{k}^{{\prime{({ini})}} - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(237)} \\ {= {M_{G}^{\prime{({ini})}}S_{k}^{\prime}M_{k}^{{\prime{({ini})}} - 1}}} & {(238)} \end{matrix}$

It is assumed that a matrix expressing the direction of the common TB_(k) coordinate system in the TBG coordinate system is Λ_(k)′^((ini)) as is obtained in the next equation.

$\begin{matrix} {\Lambda_{k}^{\prime{({ini})}} = {\Omega_{k}^{\prime{({ini})}}M_{G}^{{\prime{({ini})}} - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(239)} \\ {= {M_{G}^{\prime{({ini})}}S_{k}^{\prime}M_{G}^{{\prime{({ini})}} - 1}}} & {(240)} \end{matrix}$

As Φ_(k)′ and Λ_(k)′^((ini)) are obtained only from the initial pose, these may be obtained once at first.

(2.8.1.2. Common Coordinate System of Character Bone)

A conversion matrix between the original coordinate system of the bone and the common coordinate system is obtained using the initial state of the character.

It is assumed that a vector expressing the CBG axis in the object coordinate system is v_(G) ^((ini)). It is assumed that a vector expressing the CB_(m) axis in the object coordinate system is v_(m) ^((ini)). It is assumed that a simple rotation from v_(G) ^((ini)) to v_(m) ^((ini)) is S_(m).

It is assumed that a matrix expressing the direction of the common CB_(m) coordinate system in the object coordinate system is Ω_(m) ^((ini)). As the matrix expressing the direction of the CBG in the object coordinate system is M_(G) ^((ini)), Ω_(m) ^((ini)) can be obtained as follows by converting M_(G) ^((ini)) to be expressed in the TBG coordinate system and further multiplying S_(m) to M_(G) ^((ini)). Ω_(m) ^((ini))=A_(G) ⁻¹M_(G) ^((ini))S_(m)  (241)

Here, a matrix expressing the direction of the original CB_(m) coordinate system in the object coordinate system is M_(m) ^((ini)). Thus, a conversion matrix between M_(m) ^((ini)) and Ω_(m) ^((ini)) is previously obtained. Specifically, a matrix expressing a conversion from M_(m) ^((ini)) to Ω_(m) ^((ini)) in the original CB_(m) coordinate system is Φ_(m) as is obtained from the next equation.

$\begin{matrix} {\Phi_{m} = {\Omega_{m}^{({ini})}M_{m}^{{({ini})} - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(242)} \\ {= {A_{G}^{- 1}M_{G}^{({ini})}S_{m}M_{m}^{{({ini})} - 1}}} & {(243)} \end{matrix}$

It is assumed that a matrix expressing the direction of the common CB_(m) coordinate system in the CBG coordinate system is Λ_(m) ^((ini)) as is obtained in the next equation.

$\begin{matrix} {\Lambda_{m}^{({ini})} = {\Omega_{m}^{({ini})}M_{G}^{{({ini})} - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(244)} \\ {= {A_{G}^{- 1}M_{G}^{({ini})}S_{n}M_{G}^{{({ini})} - 1}}} & {(245)} \end{matrix}$

As Φ_(m) and Λ_(m) ^((ini)) are obtained only from the initial pose, these may be obtained once at first.

(2.8.2. Variation in Direction from Initial State)

The current state of the character is determined based on the current state of the template. First, as will be explained later, the direction of the common TB_(k) coordinate system at the current state is obtained.

It is assumed that a matrix expressing the direction of the common TB_(k) coordinate system in the object coordinate system is Ω_(k)′. Ω_(k)′ can be obtained as follows using the conversion matrix Φ_(k)′. Ω_(k)′=Φ_(k)′M_(k)′  (246)

It is assumed that a matrix expressing the direction of the common TB_(k) coordinate system in the TBG coordinate system is Λ_(k′) as is obtained in the next equation.

$\begin{matrix} {\Lambda_{k}^{\prime} = {\Omega_{k}^{\prime}M_{G}^{\prime - 1}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(247)} \\ {= {\Phi_{k}^{\prime}M_{k}^{\prime}M_{G}^{\prime - 1}}} & {(248)} \end{matrix}$

A variation of the direction of the TB_(k) from the initial state is expressed as U_(k′) as follows. U_(k)′=Λ_(k)′Λ_(k)′^((ini)−1)  (249) (2.8.3. Interpolation of Variation Amount in Direction)

Generally, the numbers of bones included in the bone group are different between the template and the character. Further, the lengths of the bones in each of the bone groups may not be the same. Thus, the variation amount U_(k)′ obtained for the template bone cannot be used as it is for the variation amount in direction of the character bone U_(m).

Thus, the variation amount in direction of the character bone U_(m) can be obtained by interpolating the variation amount U_(k)′.

FIG. 17 is a view showing a method of interpolating. FIG. 17 shows a template bone group and a character bone group of a pair (corresponding bone groups). The template bone group TBG includes three bones TB₁, TB₂ and TB₃ and the character bone group CBG includes four bones CB₁, CB₂, CB₃ and CB₄.

The bones included in the template bone group TBG and the bones included in the character bone group CBG are shown as lines and multiplied by scales so that the total lengths become 1, respectively. The total length is divided into four areas A₁ to A₄ at middle points of the bones TB₁, TB₂ and TB₃ of the template bone group TBG.

The variation amount in direction U_(m) of the character bone CB_(m) is determined based on the area A_(k) where the middle point of the character bone CB_(m) is positioned. For example, when the middle point of the character bone CB_(m) is positioned at the area A_(k), the variation amount in direction U_(m) of the character bone CB_(m) is obtained by interpolating the variation amounts in direction U_(k−1)′ and U_(k)′ (such that the variation amount at the left end of the area A_(k) becomes U_(k−1)′, and the variation amount at the right end of the area A_(k) becomes U_(k)′). Here, for the area A₁, U_(m)=U₁′, and for the area A₄, U_(m)=U₃′.

The interpolation may be performed by a known method. The method of interpolating may be as follows.

-   -   converting a matrix to a quatern ion, performing spherical         linear interpolation and converting the result to the matrix         again.     -   interpolating components of a matrix and removing distortion of         the matrix using Gram-Schmidt's orthogonalization.     -   decomposing a matrix into a bending component and a torsion         component, interpolating the components respectively, and then         composing the results.

Further, when interpolating the torsion rotations for the areas A₁ and A₄, continuity of the torsion rotations at the boundaries of the bone groups can be actualized by using the torsion rotations of the adjacent bones (bone groups).

(2.8.4. Determining Direction of Character Bone)

The direction of each of the character bones CB_(m) can be obtained as follows using U_(m) obtained by the interpolation.

First, a matrix Λ_(m) in which the direction of CB_(m) in the common coordinate system expressed in the CBG coordinate system is obtained by the next equation. Λ_(m)=U_(m)Λ_(m) ^((ini))  (250)

Then, a matrix Ω_(m) in which the direction of CB_(m) in the common coordinate system is expressed in the object coordinate system is obtained by the next equation.

$\begin{matrix} {\Omega_{m} = {\Lambda_{m}M_{G}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(251)} \\ {= {U_{m}\Lambda_{m}^{({ini})}M_{G}}} & {(252)} \end{matrix}$

Finally, a matrix M_(m) is obtained by converting CB_(m) in the common coordinate system to the original coordinate system using Φ_(m) as follows.

$\begin{matrix} {M_{m} = {\Phi_{m}^{- 1}\Omega_{m}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(253)} \\ {= {\Phi_{m}^{- 1}U_{m}\Lambda_{m}^{({ini})}M_{G}}} & {(254)} \end{matrix}$ (2.8.5. Correction of Direction of Bone Group)

When determining the direction of bones in the bone group by the above method, the direction of the bone group axis is also changed. Thus, at last, using a simple rotation from the current bone group axis to the previous bone group axis, the entirety of the bone group is rotated to set back the direction of the bone group axis.

2.9. For Bone Having Length “0”

(2.9.1. Length of Bone and Simple Rotation)

For the above method of copying, a simple rotation using a bone axis is frequently used. Thus, if a bone with a length “0” exists, the simple rotation cannot be defined and the operation cannot be processed anymore. Further, even when the bone length is not zero, if the length is extremely short, a problem in calculation may occur. Thus, the minimum allowable value e_(b) for the bone length is previously determined, and the bone length (the bone axis) is adjusted when the actual bone length is less than the minimum allowable value e_(b).

For the minimum allowable value e_(b), the value may be small enough not to cause an influence on an appearance of the pose of the model, and may be large enough to be capable of calculating (0.001 times by a height of a model, for example).

(2.9.2. Adjustment of Bone Axial Vector)

The bone length is adjusted as follows.

In the initial state, the bone lengths are detected by searching the bone groups and the bones from a parent side to a child side, and when a bone B^((zero)) whose length is less than the minimum allowable value e_(b) is found, the following operation is performed.

First, the bones in the same bone group are searched toward a child side to find a bone having a length more than or equal to the minimum allowable value e_(b). The bone found first is assumed as B^((len)). Then, the axis vector of B^((zero)) is set to have the minimum allowable value e_(b) and a direction to be B^((len)). The coordinate system of the bone is maintained.

FIG. 18 is a view showing an adjustment of an axial vector of a bone.

For example, as shown in (a) of FIG. 18, when the bone length B₂ is less than e_(b), the bone length B₂ becomes as shown in (b) by the adjustment.

When a bone having a length more than or equal to e_(b) cannot be found, a parent bone in the same bone group is used as B^((len)). When the lengths of all of the bones in the bone group are less than e_(b), an axis in the parent bone group is used instead of B^((len)) axis. When a parent bone group does not exist, a previously determined vector is used.

Here, this adjusted bone axis is used only for calculations in determining the pose of the character, and is not used when updating the pose of the character at last. Thus, when updating the pose of the character, only the direction (rotation) of the adjusted bone is used and the bone length is not changed.

2.10. Appendix A: Simple Rotation

FIG. 19 is a view showing a simple rotation.

The “simple rotation” is defined as follows. A simple rotation from v_(a) to v_(b) where v_(a) and v_(b) are two arbitrary vectors (or line segments with arrows respectively) having a common starting point is defined to comply with the following:

-   -   the rotation is around an axis which is perpendicular to a plane         formed by v_(a) and v_(b)     -   v_(a) overlaps v_(b) by the rotation     -   when v_(a) and v_(b) are in the same direction (v_(a)=kv_(b),         k>0), no rotation is performed (expressed by a unit matrix as         the rotation matrix)     -   when v_(a) and v_(b) are opposite (v_(a)=kv_(b), k<0), it is         impossible to define.

The pose or movement of a skeleton model to be controlled can be controlled with a simple method. The individual constituents of the model control system 800 may be embodied by arbitrary combinations of hardware and software, typified by a CPU of an arbitrary computer, memory, a program loaded in the memory so as to embody the constituents illustrated in the drawings, storage units for storing the program such as a hard disk, and an interface for network connection. It may be understood by those skilled in the art that methods and devices for the embodiment allow various modifications.

Although a preferred embodiment of the method of controlling model has been specifically illustrated and described, it is to be understood that minor modifications may be made therein without departing from the spirit and scope of the invention as defined by the claims.

The present invention is not limited to the specifically disclosed embodiments, and variation and modifications may be made without departing from the scope of the present invention.

The present application is based on Japanese Priority Application No. 2011-266383 filed on Dec. 6, 2011, the entire contents of which are hereby incorporated by reference. 

What is claimed is:
 1. A method of controlling a target skeleton model including a plurality of bones that are connected by joints, by a computer, comprising: displaying an image of the target skeleton model on a screen for an operator; accepting an instruction of changing a pose or a movement of the target skeleton model displayed on the screen from the operator via an interface; applying the instruction of changing the pose or the movement of the target skeleton model to a standard skeleton model including a plurality of bones that are connected by joints, control data for joint angles to apply a limitation to a pose or a movement of the standard skeleton model being attached to the standard skeleton model, the number of bones of the standard skeleton being different from the number of bones of the target skeleton model, the plurality of bones of the standard skeleton model being classified into a plurality of bone groups of the standard skeleton model, and the plurality of bones of the target skeleton model being classified into a plurality of bone groups of the target skeleton model such that the plurality of bone groups of the standard skeleton model being in correspondence with the plurality of bone groups of the target skeleton model, respectively, in one-to-one relationship, for at least one of the bone groups of the standard skeleton model, the number of the bones included in the respective bone group of the standard skeleton model being different from the number of the bones included in the corresponding bone group of the target skeleton model that is in correspondence with the bone group of the standard skeleton model, and the instruction of changing the pose or the movement of the target skeleton model being converted to an instruction of changing the pose or the movement of the standard skeleton model based on the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model; determining a pose or a movement of the standard skeleton model based on the instruction and the control data for joint angles attached to the standard skeleton model; and determining a pose or a movement of the target skeleton model such that the pose or the movement of the target skeleton model follows the pose or the movement of the determined standard skeleton model based on the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model, respectively, wherein the determining the pose or the movement of the target skeleton model includes specifying one of the bone groups of the standard skeleton model and a corresponding one of the bone groups of the target skeleton model based on the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model, copying a direction of the specified bone group of the standard skeleton model into the specified bone group of the target skeleton model, and copying a direction of each of the bones included in the specified bone group of the standard skeleton model into the corresponding bones included in the specified bone group of the target skeleton model.
 2. The method of controlling the target skeleton model according to claim 1, wherein the control data for joint angles includes a setting of an angle limitation or a hardness in bending for each of the joints of the standard skeleton model.
 3. The method of controlling the target skeleton model according to claim 1, wherein the pose or the movement of the target skeleton model is determined based on the relationship between the pose or the movement of the standard skeleton model at an initial state and the pose or the movement of the target skeleton model at an initial state.
 4. The method of controlling the target skeleton model according to claim 1, wherein when the bone group includes a plurality of bones, the angle of each of joints connecting the plurality of bones is determined based on the angles of joints at a starting point and an end point.
 5. The method of controlling the target skeleton model according to claim 1, wherein the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model is obtained by comparing structures of the standard skeleton model and the target skeleton model or as instructed by an operator.
 6. The method of controlling the target skeleton model according to claim 1, wherein the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model is obtained by pattern recognition.
 7. A non-transitory computer-readable recording medium having recorded thereon a program that causes a computer to execute the method of claim
 1. 8. The method of controlling the target skeleton model according to claim 1, wherein in the determining the pose or the movement of the target skeleton model, determining the pose or the movement of each of the bone groups of the target skeleton model to follow the corresponding bone group of the standard skeleton model such that the pose or the movement of the target skeleton model follows the pose or the movement of the determined standard skeleton model based on the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model, respectively.
 9. The method of controlling the target skeleton model according to claim 1, wherein the target skeleton model is a new generated skeleton model and the standard skeleton model is a previously prepared skeleton model provided for applying a limitation to the pose or the movement of the target skeleton model based on the control data for join angles.
 10. The method of controlling the target skeleton model according to claim 1, wherein control data for joint angles is not attached to the target skeleton model.
 11. The method of controlling the target skeleton model according to claim 1, wherein after performing the determining the pose or the movement of the standard skeleton model and the determining the pose or the movement of the target skeleton model, a new pose or movement of the target skeleton model is displayed on the screen for the operator.
 12. The method of controlling the target skeleton model according to claim 1, wherein each of the bone groups is an assembly of linearly connected one or more bones.
 13. A method of controlling a target skeleton model including a plurality of bones, comprising: using a standard skeleton model including another plurality of bones the number of which is different from the number of the bones of the target skeleton model, classifying the plurality of bones of the standard skeleton model into a plurality of bone groups of the standard skeleton model, and classifying the plurality of bones of the target skeleton model into a plurality of bone groups of the target skeleton model such that the plurality of bone groups of the standard skeleton model become in correspondence with the plurality of bone groups of the target skeleton model, respectively, in one-to-one relationship, for at least one of the bone groups of the standard skeleton model, the number of the bones included in the respective bone group of the standard skeleton model being different from the number of the bones included in the corresponding bone group of the target skeleton model that is in correspondence with the bone group of the standard skeleton model; determining a pose or a movement of the standard skeleton model based on an instruction; determining a pose or a movement of the target skeleton model such that the pose or the movement of the target skeleton model follows the pose or the movement of the determined standard skeleton model by specifying one of the bone groups of the standard skeleton model and a corresponding one of bone groups of the target skeleton model based on the correspondence of the bone groups of the standard skeleton model and the bone groups of the target skeleton model and the pose or the movement of the standard skeleton model, copying a direction of the specified bone group of the standard skeleton model into the specified bone group of the target skeleton model such as to follow the direction of the specified bone group of the standard skeleton model, and copying a direction of each of the bones included in the specified bone group of the standard skeleton model into the corresponding bones included in the specified bone group of the target skeleton model.
 14. A non-transitory computer-readable recording medium having recorded thereon a program that causes a computer to execute the method of claim
 13. 15. The method of controlling the target skeleton model according to claim 13, wherein the target skeleton model is a new generated skeleton model and the standard skeleton model is a previously prepared skeleton model having a function to apply a limitation to the pose or the movement of the target skeleton model.
 16. The method of controlling the target skeleton model according to claim 13, wherein the bones of the standard skeleton model are connected by joints and the bones of the target skeleton model are connected by joints, and wherein control data for joint angles is attached to the standard skeleton model and control data for joint angles is not attached to the target skeleton model.
 17. The method of controlling the target skeleton model according to claim 16, further comprising: displaying an image of the target skeleton model on a screen for an operator; accepting an instruction of changing a pose or a movement of the target skeleton model by the operator via an interface; converting the instruction of changing the pose or the movement of the target skeleton model to an instruction of changing a pose or a movement of the standard skeleton model based on the correspondence of the bone groups of the target skeleton model and the bone groups of the standard skeleton model, respectively; and determining a pose or a movement of the standard skeleton model based on the converted instruction and the control data for joint angles attached to the standard skeleton model, wherein after performing the determining the direction of each of the bones included in the specified group of bones of the target skeleton model, a new pose or movement of the target skeleton model is displayed on the screen for the operator.
 18. The method of controlling the target skeleton model according to claim 13, wherein each of the bone groups is an assembly of linearly connected one or more bones. 